Modeling Software Quality Evaluation Using the Analytic Hierarchy Process

This paper proposes a modified software quality evaluation method based on products comparison, judging by user perceived satisfaction in terms of quality. The focused evaluation model is formulated on the bases of the ISO 9126 quality model as a generally accepted standard and ISO 12119 for software package, and of the Analytic Hierarchy Process (AHP) which is appropriate for comparing paired objects with subjective judgment. Getting priorities or weights vector, the AHP/ANP (Analytic Network Process) is usefully accepted when the evaluation depends on subjective judgment and/or expert knowledge. An outstanding advantage of the AHP/ANP is pairwise comparisons, which can covers subjective and fuzzy evaluation. However, it is not uncommon that an evaluator does not empirically experience all of multiple software products in a same category and that paired comparison is known to be more valid than independently single alternative evaluation. This leads to the limited pairwise comparisons among alternatives from each evaluator’s experience and judgment even though the method is the way to raise the comparison validity. Then, the required pairwise comparison matrix is proposed by gathering such limited comparisons, called partial pairwise comparison matrix, from all evaluators who has different products experience or knowledge. The proposed model is empirically exemplified with popular software products. Only reliable data, identified by screening with the consistency ratio of the AHP, are analyzed. This paper discusses to estimate missing value and rank order with the result using the compatibility index in the AHP.